Image processing apparatus, projection display apparatus, video display system, image processing method, and computer readable storage medium

ABSTRACT

An image processing apparatus processes image data corresponding to a display image. The image processing apparatus includes M image processing sections which have a function of processing image data of M (where M is an integer equal to or greater than 2) images segmented from the image and an image processing control section which, when an image corresponding to input image data is an image which has a resolution being of size to be processable by N (where N is an integer equal to or smaller than M−1) image processing sections, causes L image processing sections from among the M image processing sections to process image data corresponding to L (where L is an integer equal to or greater than N+1 and equal to or smaller than M) partial images segmented from image data.

The entire disclosure of Japanese Patent Application No. 2010-43978,filed Mar. 1, 2010 is expressly incorporated by reference herein.

BACKGROUND

1. Technical Field

The present invention relates to an image processing technique whichsegments image data corresponding to a display image into multiplepieces of image data and processes multiple pieces of image data.

2. Related Art

In recent years, a video processing apparatus which is mounted in avideo display instrument, such as a high-resolution projector, a liquidcrystal television, or a plasma television, segments image datacorresponding to an input display image (hereinafter, referred to asdisplay image data) into multiple pieces of image data and processesmultiple pieces of image data in parallel. Specifically, a plurality ofimage processing sections, each of which processes segmented image data(hereinafter, referred to as partial image data), process partial imagedata in parallel. Example of such a technique is described inJP-A-2009-111969.

However, in the case of a video processing apparatus which uses a methodof segmenting multiple pieces of image data and processing multiplepieces of image data in parallel, regardless of whether input displayimage data has high resolution or low resolution, in general, the samedata quantity of image data is processed by the single image processingsection. That is, high-resolution display image data is processed by alarge number of image processing sections, and when low-resolutiondisplay image data is input, a small number of image processing sectionsare used to process low-resolution display image data depending on theratio of lowering of resolution. In processing low-resolution imagedata, in addition to image processing of high-resolution image data,resolution conversion processing (for example, pixel interpolation bybi-cubic convolution or bilinear interpolation) is performed. As aresult, in processing low-resolution image data, there is a problem inthat a heavy load is imposed on the single image processing section.

SUMMARY

An advantage of some aspects of the invention is to solve at least apart of the problems described above and the invention can beimplemented as the following aspects or application examples.

An image processing apparatus in one application example processes imagedata corresponding to a display image. The image processing apparatusincludes an input section which is configured to input at least imagedata corresponding to an image having a reference resolution P which isa resolution as a reference, the resolution representing the number ofconstituent pixels, M image processing sections which have a function ofprocessing image data of M (where M is an integer equal to or greaterthan 2) images segmented from the image having the reference resolutionP, an image processing control section which, when an imagecorresponding to input image data is the image having the referenceresolution P, causes the M image processing sections to process imagedata corresponding to the M partial images segmented from image dataand, when an image corresponding to input image data is an image whichhas a resolution Q lower than the reference resolution P and is of sizeto be processable by N (where N is an integer equal to or smaller thanM−1) image processing sections, causes L image processing sections fromamong the M image processing sections to process image datacorresponding to L (where L is an integer equal to or greater than N+1and equal to or smaller than M) partial images segmented from imagedata, and an image composition section which reconstructs the displayimage on the basis of image data corresponding to the partial imagessubjected to image processing by the M or L image processing sections.

With this image processing apparatus, when image data corresponding tothe display image having the resolution Q lower than the referenceresolution P is processed by the image processing sections as partialimage data, instead of decreasing the number of image processingsections for processing depending on the ratio of lowering ofresolution, a large number of image processing sections, specifically,L, that is, “N+1” or more image processing sections process image datafor the ratio of lowering of resolution.

Usually, if the resolution of image data corresponding to a displayimage is lowered, it is necessary to perform processing according to thelowering of resolution. For example, one of the processing is resolutionconversion processing. With regard to resolution conversion, imageinterpolation processing by bilinear interpolation or bi-cubicconvolution is performed. If the number of image processing sections forprocessing decreases depending on the lowering of resolution, a heavyprocessing load is imposed on the single image processing sectioncompared to a case where the reference resolution P is processed. Thatis, the resolution conversion processing is performed in addition toimage processing which is performed by the single image processingsection in the case of the reference resolution P.

Meanwhile, in the image processing apparatus according to theapplication example of the invention, a large number of image processingsections, that is, “N+1” or more image processing sections perform imageprocessing for the ratio of lowering of resolution. Thus, it is possibleto reduce a load on the single image processing section compared to animage processing apparatus in which the number of image processingsections for processing decreases depending on the ratio of lowering ofresolution. Therefore, it is possible to reduce the performance of thesingle image processing section and consequently to achieve reduction incost.

In one application example, a plurality of input sections may beprovided to input image data corresponding to the display image asmultiple pieces of segmented image data.

With this image processing apparatus, a plurality of input sections areprovided, such that image data can be input in parallel from a pluralityof input sections in a segmented form.

In one application example, the L image processing sections may includea first image processing section which, before the image processing,receives image data input by the input section, segments received imagedata into a predetermined number of pieces of image data, reserves atleast one piece of segmented image data for use in processing, anddistributes unreserved segmented image data to other image processingsections, and a second image processing section which receives imagedata distributed by the first image processing section.

With this image processing apparatus, when a plurality of imageprocessing sections process input image data, the first image processingsection receives and segments image data, and distributes segmentedimage data to the second image processing section. Therefore, when theinput section inputs image data, it is not necessary to take the form ofinput such that image data is input to the respective image processingsections.

In one application example, the second image processing section maysegment image data received from the first processing section into apredetermined number of pieces of image data, reserve at least one pieceof segmented image data for use in processing, and distribute unreservedsegmented image data to image processing sections other than the firstand second image processing sections, and the L image processingsections may include a third image processing section which receivesimage data distributed by the second image processing section.

With this image processing apparatus, the second image processingsection segments received image data and distributes segmented imagedata to the third image processing section. Therefore, when the inputsection inputs image data, it is not necessary to take the form of inputsuch that image data is input to the respective image processingsections.

In one application example, M and L may be the same number.

With this image processing apparatus, even when the resolution of inputimage data is lowered, image data is processed by the M image processingsections.

In one application example, the reference resolution P may be “8K4K”.

This image processing apparatus copes with so-called super high-visionimage data.

A projection display apparatus in one application example projects aprojected image on a projection surface on the basis of image datacorresponding to a display image. The projection display apparatusincludes the above-described image processing apparatus.

With this projection display apparatus, the above-described imageprocessing apparatus is provided, reducing a load of processing on thesingle image processing section. Therefore, it is possible to reduce theperformance of the single image processing section in the projectiondisplay apparatus and consequently achieving reduction in cost.

A video display system in one application example includes a videodisplay apparatus which includes the above-described image processingapparatus and displays an image on the basis of image data correspondingto a display image, and a storage which stores image data correspondingto the display image and inputs image data corresponding to the displayimage to the video display apparatus.

With this video display system, the video display apparatus includes theabove-described image processing apparatus, reducing the performance ofthe single image processing section in the video display apparatus andconsequently achieving reduction in cost.

An image processing method in one application example processes imagedata corresponding to a display image and processes at least image datacorresponding to an image having a reference resolution P which is aresolution as a reference, the resolution representing the number ofconstituent pixels. The image processing method includes, when imagedata corresponding to the input display image is the image having thereference resolution P, segmenting image data into image datacorresponding to M (where M is an integer equal to or greater than 2)partial images having a resolution R and processing segmented image datain parallel, when image data corresponding to the input display imagehas a resolution Q lower than the reference resolution P, and one of N(where N is an integer equal to or smaller than M−1) segmented partialimages has a resolution equal to or lower than a resolution R,segmenting image data into L (where L is an integer equal to or greaterthan N+1 and equal to or smaller than M) pieces of image data andprocessing image data corresponding to the partial images in parallel,and reconstructing the display image on the basis of processed imagedata corresponding to the M or L partial images.

With this image processing method, in processing image datacorresponding to the display image having the resolution Q lower thanthe reference resolution P, instead of decreasing the number of piecesof segmented image data depending on the ratio of lowering ofresolution, image data is segmented into a large number of pieces ofimage data, specifically, L, that is, “N+1” or more pieces of image datafor the ratio of lowering of resolution and segmented image data isprocessed in parallel. Therefore, while image processing is accompaniedby resolution conversion, it is possible to reduce a load of processinga single partial image.

A computer readable storage medium storing a computer program in oneapplication example causes a computer to realize an image processingfunction of processing image data corresponding to a display image andprocessing at least image data corresponding to an image having areference resolution P which is a resolution as a reference, theresolution representing the number of constituent pixels. The computerprogram causes the computer to execute, when image data corresponding tothe input display image is the image having the reference resolution P,segmenting image data into image data corresponding to M (where M is aninteger equal to or greater than 2) partial images having a resolution Rand processing segmented image data in parallel, when image datacorresponding to the input display image has a resolution Q lower thanthe reference resolution P, and one of N (where N is an integer equal toor smaller than M−1) segmented partial images has a resolution equal toor lower than a resolution R, segmenting image data into L (where L isan integer equal to or greater than N+1 and equal to or smaller than M)pieces of image data and processing image data corresponding to thepartial images in parallel, and reconstructing the display image on thebasis of processed image data corresponding to the M or L partialimages.

With this computer readable storage medium, in processing image datacorresponding to the display image having the resolution Q lower thanthe reference resolution P, instead of decreasing the number of piecesof segmented image data depending on the ratio of lowering ofresolution, the computer realizes the function of segmenting image datainto a large number of pieces of image data, specifically, L, that is,“N+1” or more pieces of image data for the ratio of lowering ofresolution and processing segmented image data in parallel. Therefore,while image processing is accompanied by resolution conversion, it ispossible to reduce a load of processing a single partial image.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the accompanyingdrawings, wherein like numbers reference like elements.

FIG. 1 is an explanatory view illustrating a video display system.

FIG. 2 is an explanatory view illustrating the configuration of aprojector.

FIG. 3 is an explanatory view showing the configuration of a videoprocessing apparatus.

FIG. 4 is a block diagram showing the internal configuration of a sixthimage processing section.

FIG. 5 is a block diagram schematically showing processing in the sixthimage processing section.

FIG. 6 is an explanatory view schematically showing the flow of videoprocessing in the video processing apparatus.

FIG. 7 is an explanatory view illustrating ambient pixel data conversionprocessing.

FIG. 8 is an explanatory view illustrating a video display system.

FIG. 9 is an explanatory view illustrating the flow of input mode changeprocessing.

FIGS. 10A and 10B are explanatory views illustrating an input method anda processing method of partial image data when an input mode is changed.

FIGS. 11A and 11B are flowcharts illustrating the flow of imageprocessing.

FIG. 12 is an explanatory view illustrating an input method and aprocessing method of partial image data when an input mode is changed toa 2K1K mode.

FIG. 13 is an explanatory view illustrating Modification 3.

DESCRIPTION OF EXEMPLARY EMBODIMENT

An embodiment of the invention will be described on the basis ofExamples.

A First Example A1 Configuration of Video Display System

FIG. 1 is an explanatory view illustrating a video display system 10 ofFirst Example. The video display system 10 includes a projector PJ whichprojects and displays a video, a screen SC which displays the videoprojected by the projector PJ, computers PC1 to PC16, video storages St1to St16 serving as a storage device, and a master computer PCM whichcontrols the data output from the computers PC1 to PC16. The projectorPJ is connected to the computers PC1 to PC16 through a data transmissioncable, receives video data having a plurality of types of resolution,performs image processing, and projects and displays a display image onthe screen SC. The computers PC1 to PC16 are connected to the mastercomputer PCM through a LAN (Local Area Network) cable. The videostorages St1 to St16 are connected to the computers PC1 to PC16 througha data communication cable.

As shown in FIG. 1, each of the video storages St1 to St16 stores 16pieces of partial image data DIn1 to DIn16 in total obtained byquartering display image data DIn0 for one screen of a video projectedand displayed on the screen SC by the projector PJ in the horizontal andvertical directions by the number of frames of the video. DIn1 to DIn16are image data of size equivalent to so-called full HD. Image data ofthis size is generally called “2K1K” (resolution 1920×1080). The term“resolution” used herein means the number of pixels constituting imagedata corresponding to one image. In the case of the video display system10, display image data DIn0 includes 2K1K partial image data arranged byfour in the vertical and horizontal directions. Display image data DIn0is generally image data having a resolution of called “8K4K”.

Images corresponding to partial image data DIn1 to DIn16 are processedby a video processing apparatus 100 in the projector PJ described below.Image data processed by the video processing apparatus 100 is projectedand displayed as partial image data DIn1 to DIn16 on the screen SCthrough the projector PJ, such that display image data DIn0 which is theimage for one screen is displayed on the screen SC. Partial image dataDIn1 to DIn16 are synchronously displayed on the screen SC continuouslyin a time series manner, such that a video as a motion image or a stillimage is displayed on the screen SC.

The computers PC1 to PC16 receive a synchronization signal from themaster computer PCM through a LAN, synchronously read partial image dataDIn1 to DIn16 from the video storages St1 to St16, and input partialimage data DIn1 to DIn16 to the projector PJ. The master computer PCM isa computer for controlling the read timing at which the computers PC1 toPC16 read partial image data DIn1 to DIn16 from the video storages St1to St16. If a user manipulates the master computer PCM to issue aninstruction to start video display, the master computer PCM outputs asynchronization signal to the respective computers PC1 to PC16, and thenpartial image data DIn1 to DIn16 are started to be read from the videostorages St1 to St16. In this example, partial image data DIn1 to DIn16are digital data regardless of whether partial image data DIn1 to DIn16are stored in the video storage or transmitted to the projector PJ.

Next, the configuration of the projector PJ will be described. FIG. 2 isan explanatory view illustrating the configuration of the projector PJ.The projector PJ includes a CPU 20, a ROM 21, a RAM 22, an LED/IF 23, anLED 24, connection terminals 25, a liquid crystal panel driving section26, an illumination optical system 27, a liquid crystal panel 28, aprojection optical system 29, a video processing apparatus 100, amanipulation section Ctr, and an internal bus 30. The CPU 20 has acontrol function of performing overall control of the projector PJ. Inthis example, the CPU 20 includes an input mode control section 65 inaddition to the control function of performing overall control of theprojector PJ. The manipulation section Ctr includes a panel forselecting “8K4K mode” which is set when input display image data DIn0 is8K4K, “4K2K mode” which is set when input display image data DIn0 is4K2K, and “2K1K mode” which is set when display image data DIn0 is 2K1K.These modes are called “input mode”. The user sets the input modethrough the manipulation section Ctr in accordance with the resolutionof input display image data DIn0. In the case of the video displaysystem 10, the user sets “8K4K mode” as the input mode.

The connection terminals 25 are terminals which connect the computersPC1 to PC16 and the projector PJ to each other and receive partial imagedata DIn from the computers PC1 to PC16. The video processing apparatus100 processes partial image data DIn input through the connectionterminals 25 and outputs processed video data to the liquid crystalpanel driving section 26. The liquid crystal panel driving section 26drives the liquid crystal panel 28 on the basis of input video data. Theliquid crystal panel 28 is a transmissive liquid crystal panel whichvisualizes a generated signal by the liquid crystal panel drivingsection 26, and modulates light irradiated from the illumination opticalsystem 27 to emit light (projection light) necessary for projectiontoward the screen SC. The liquid crystal panel 28 may be a light valveusing a DMD (Digital Micromirror Device, Registered Trademark), insteadof a transmissive liquid crystal panel. The projection optical system 29includes a projection lens or the like (not shown) which projects lightirradiated from the illumination optical system 27 through the liquidcrystal panel 28 on the screen SC on a magnified scale.

Next, the configuration of the video processing apparatus 100 in theprojector PJ will be described. FIG. 3 is an explanatory view showingthe configuration of the video processing apparatus 100. The videoprocessing apparatus 100 includes video input sections 201 to 216 whichinput partial image data DIn1 to DIn16 input from the computers PC1 toPC16 through the connection terminals 25 to the video processingapparatus 100, first image processing section 301 to 16th imageprocessing section 316 as 16 image processing sections which processinput partial image data DIn1 to DIn16 in parallel, a video compositionsection 40 which composes image data DIn1 to DIn16 corresponding topartial image data processed by the image processing sections inparallel to generate image data of one screen, a video output section 50which outputs composed display image data DIn0 as an output signal tothe liquid crystal panel driving section 26, and a timing instructionsection 60. The image processing sections 301 to 316 are connected toeach other through a data exchange bus 45.

The 16 image processing sections 301 to 316 receive an instruction fromthe timing instruction section 60 and exchange ambient pixel datanecessary for image processing in the image processing sections 301 to316 with each other. Ambient pixel data will be described below. Theimage processing sections perform processing with the lower two-digitnumber of the reference numeral attached to each image processingsection associated with the number of the reference numeral attached toeach piece of partial image data such that the first image processingsection 301 processes DIn1 from among segmented partial image data DIn1to DIn16 and the second image processing section 302 processes DIn2. Theimage processing sections receive information (hereinafter, alsoreferred to as input mode signals) regarding the input mode set by theuser using the manipulation section Ctr through the input mode controlsection 65.

The timing instruction section 60 transmits a synchronization outputsignal for synchronously outputting partial image data DIn processed bythe respective image processing sections from the respective imageprocessing sections to the video composition section 40 to the imageprocessing sections 301 to 316. The synchronization output signal isgenerated on the basis of, for example, a dot clock or a synchronizationsignal, such as a horizontal synchronization signal and a verticalsynchronization signal, and is transmitted to the image processingsections 301 to 316. In the following description, the configuration ofthe image processing section will be described focusing on theconfiguration of the sixth image processing section 306.

FIG. 4 is a block diagram showing the internal configuration of thesixth image processing section 306. The sixth image processing section306 includes a CPU 71 which has a function as a digital signal processor(DSP), a ROM 73 which stores an operating program or the like, a RAM 75which is used as a work area, a frame memory 80 which has a storagecapacity slightly greater than image data segmented from display imagedata DIn0, that is, partial image data DIn6, an input interface 81 whichreceives partial image data DIn6 from the video storage St6, an outputinterface 83 which outputs partial image data DIn6 to the videocomposition section 40, an instruction input interface 85 which receivesa timing signal from the timing instruction section 60, and a mode inputinterface 87 which receives an input mode signal from the input modecontrol section 65. The CPU 71 is an exclusive-use processor whichcontrols the overall operation of the sixth image processing section 306and, in particular, can access the frame memory 80 at high speed toperform predetermined image processing (filtering). The function of theCPU 71 may be realized by an FPGA (Field Programmable Array) or an LSIfor image processing only.

Next, the functional configuration of each image processing section willbe described. FIG. 5 is a block diagram schematically showing processingin the sixth image processing section 306. The sixth image processingsection 306 functionally includes a segmented video input section 361, adata exchange section 362, a frame memory control section 363, a framememory 364, a filtering section 365, a segmented image output section366, and a data distribution control section 367. Actually, theoperation of each block is realized when the CPU 71 executes apredetermined program. The details of the respective functional sectionswill be described below.

A2 Video Processing

Next, description will be provided as to video processing in the 8K4Kmode by the video processing apparatus 100. FIG. 6 is an explanatoryview schematically showing the flow of video processing in the videoprocessing apparatus 100. The video processing starts when partial imagedata DIn1 to DIn16 are input from the video storages St1 to St16 (seeFIG. 1) to the video input sections 201 to 216.

Partial image data DIn1 to DIn16 are respectively input from the videoinput sections 201 to 216 to the image processing sections 301 to 316through the segmented video input sections (see FIG. 4) (Step S120). Theframe memory control section of each image processing section storesinput partial image data DIn in the frame memory. If the storage ofpartial image data DIn in the frame memory has been completed, the framememory control section notifies the timing instruction section 60 of thestorage having been completed. The timing instruction section 60analyzes the situation of the storage of partial image data DIn in therespective image processing sections 301 to 316 and, when it isdetermined that the input of all pieces of partial image data DIn1 toDIn16 to the image processing sections has been completed (Step S130:Yes), instructs the data exchange sections of the respective imageprocessing sections to start data exchange. If the instruction to startdata exchange is received from the timing instruction section 60, eachdata exchange section performs ambient pixel data exchange processingfor exchanging ambient pixel data, which is necessary for processingpartial image data to be processed by the corresponding image processingsection, with a data exchange section in a predetermined imageprocessing section (Step S140). The ambient pixel data exchangeprocessing will be described below in detail. From the viewpoint thatimage data is sequentially received, data exchange may be sequentiallystarted from data exchange between the image processing sections whichcan perform data exchange. Meanwhile, in this example, as shown in StepS130, for ease of understanding of the invention, data exchange isperformed after all the first to 16th image processing sections 301 to316 have received image data.

If the data exchange sections of the respective image processingsections have ended the exchange of ambient pixel data, each framememory control section outputs partial image data DIn stored in theframe memory and ambient pixel data acquired by the ambient pixel dataexchange processing to the corresponding filtering section, and thefiltering section performs filtering using the two pieces of data (StepS150). If the filtering has ended, the filtering section outputsprocessed data to the video composition section 40 through thecorresponding segmented image output section. At this time, thesegmented image output sections of the respective image processingsections 301 to 316 output partial image data DIn1 to DIn16 subjected tothe image processing to the video composition section 40 in parallel.

The video composition section 40 performs image composition processingwhich, for partial image data DIn1 to DIn16 received from the segmentedimage output sections in parallel, includes arrangement decisionprocessing for sorting the arrangement of partial image data andrearranging image data so as to be displayed as display image data DIn0when partial image data is displayed synchronously (Step S160). Afterthe image composition processing, the video composition section 40transmits partial image data to the video output section 50 (Step S170).The video output section 50 receives rearranged partial image data DIn1to DIn16 from the video composition section 40 and synchronously outputspartial image data as an output signal to the liquid crystal paneldriving section of the liquid crystal projector (Step S180). This imageprocessing is repeatedly performed for input partial image data DIn1 toDIn16, such that the video processing apparatus 100 performs the imageprocessing. With the above, the video processing in the 8K4K mode by thevideo processing apparatus 100 has been described.

A3 Ambient Pixel Data Exchange Processing

Next, the above-described ambient pixel data exchange processing (seeFIG. 6: Step S140) will be described. First, ambient pixel data will bedescribed. As a specific example, FIG. 7 is an explanatory viewillustrating ambient pixel data which is necessary when the sixth imageprocessing section 306 performs filtering of partial image data DIn6. Bymeans of a filtering matrix of 5 rows×5 columns centered on pixels(hereinafter, also referred to as attention pixels) to be processed inpartial image data DIn6, the filtering section 365 performs filteringfor the attention pixels while referencing image data by two pixelsaround the attention pixels. Specifically, filtering is performed by aLaplacian filter or a median filter for edge enhancement or denoising,or a filter for image processing, such as a Kalman filter. In performingthe filtering, when pixels inward of four sides (upper side, lower side,left side, and right side) in total on the upper and lower sides in thevertical direction and the left and right sides in the horizontaldirection of partial image data DIn6 are to be processed as attentionpixels by two pixels, pixels which are referenced for the filteringinclude pixels in partial image data DIn1 to DIn3, DIn5, DIn7, and DIn9to DIn11 which are partial image data around partial image data DIn6.Thus, the sixth image processing section 306 has to acquire ambientpixel data shown in FIG. 7 as ambient pixel data from partial image dataDIn1 to DIn3, DIn5, DIn7, and DIn9 to DIn11 around partial image dataDIn6. The data exchange section 362 of the sixth image processingsection 306 acquires ambient pixel data from the image processingsections 301 to 303, 305, 307, and 309 to 311, which input partial imagedata DIn1 to DIn3, DIn5, DIn7, and DIn9 to DIn11, through the bus 45 bythe ambient pixel data exchange processing (see FIG. 6: Step S140). Inthis way, the ambient pixel data exchange processing is performed. Afterthe ambient pixel data exchange processing, each image processingsection performs filtering of partial image data DIn with reference toacquired ambient pixel data and outputs processed partial image data tothe video composition section 40.

A4 Input Mode Change Processing

Next, input mode change processing which is performed to change theresolution of display image data DIn0 input to the projector PJ will bedescribed. In the above-described video display system 10 (see FIG. 1),display image data DIn0 having a resolution of 8K4K is segmented into2K1K partial image data DIn1 to DIn16 and partial image data DIn1 toDIn16 are respectively input to the image processing sections 301 to316. Next, a case where display image data DIn0 is input while theresolution is changed to 4K2K is taken into consideration. This case isshown in FIG. 8. FIG. 8 is an explanatory view showing a video displaysystem 10 a when 4K2K data is input as display image data DIn0. Therespective constituent elements of the video display system 10 a are thesame as those in the video display system 10, thus description thereofwill be omitted. There is a difference in that the number of computersconnected to the projector PJ are four computers PC1 to PC4, the numberof video storages are four video storages St1 to St4, and the resolutionof DIn0 is changed from 8K4K to 4K2K. Since display image data DIn0 is4K2K, in this case, four pieces of 2K1K partial image data, that is,DIn1 to DIn4 are input to the projector PJ. DIn1 to DIn4 aresequentially input from the video storages St1 to St4 to the projectorPJ through the computers PC1 to PC4.

The user carries out change setting of the input mode from themanipulation section Ctr of the projector PJ in accordance with thechange of the input mode. FIG. 9 is an explanatory view illustrating theflow of input mode change processing in the projector PJ when the inputmode is changed. If the user changes the input mode through themanipulation section Ctr, the input mode control section 65 transmits aninput mode signal according to the input mode set by the user to thedata distribution control section (see FIG. 5) in each image processingsection (Step S210). If the input mode signal is received, the datadistribution control section of each image processing section changes aninput method and a processing method of partial image data DIn input toeach image processing section (Step S220). The details of the changedinput method and processing method of partial image data will bedescribed below in detail. Thereafter, the input mode control section 65turns on the LED 24 in the connection terminal 25, which connects theprojector PJ and the corresponding computer to each other, through theLED/IF 23 (Step S230). The LED 24 is an LED which informs the user of towhich of the 16 connection terminals 25 of the projector PJ thecomputers PC1 to PC4 will be connected. In this example, in the case ofthe 4K2K mode, the LEDs 24 of the connection terminals corresponding tothe video input sections 201, 203, 209, and 211 are turned on. Thereason why the LEDs 24 are turned on irregularly will be describedbelow.

The user confirms the turned-on LEDs 24 and connects the computers PC1to PC4 to the connection terminals 25 (the connection terminalscorresponding to the video input sections 201, 203, 209, and 211) withthe LEDs 24 turned on. Thereafter, if the user issues an instruction tostart video display through the master computer PCM, partial image dataDIn1 to DIn4 are input from the video storages St1 to St4 to theprojector PJ. As described above, the resolution of partial image dataDIn1 to DIn4 is 2K1K.

Next, the input method and processing method of partial image data inthe respective image processing sections along with the input of partialimage data DIn1 to DIn4 to the projector PJ will be described. FIGS. 10Aand 10B are explanatory views illustrating the input method andprocessing method of partial image data DIn1 to DIn4 which are changedwith the change of the input mode. In comparison with the input methodand processing method in the 8K4K mode, FIG. 10A is an explanatory viewillustrating the input method and processing method in the 8K4K mode,and FIG. 10B is an explanatory view illustrating the input method andthe processing method in the 4K2K mode. First, the input method andprocessing method of partial image data DIn in the 8K4K mode as acomparative example will be described with reference to FIG. 10A.

In the 8K4K mode, as described in the video display system 10 (see FIG.1), partial image data DIn1 to DIn16 are respectively input from thecomputers PC1 to PC16 to the image processing sections 301 to 316through the video input sections 201 to 216. FIG. 10A shows this case.For ease of understanding, the image processing sections 301 to 316 areshown in a matrix. At this time, the resolution of DIn1 to DIn16 inputto the image processing sections 301 to 316 is 2K1K.

In contrast, FIG. 10B shows a case where the input mode is changed tothe 4K2K mode. In the 4K2K mode, as described in the video displaysystem 10 a of FIG. 8, DIn1 to DIn4 are input from the computers PC1 toPC4 to the projector PJ. At this time, the user connects the computerPC1 to the video input section 201, the computer PC2 to the video inputsection 203, the computer PC3 to the video input section 209, and thecomputer PC4 to the video input section 211 with the change of the inputmode. Thus, 2K1K partial image data DIn1 to DIn4 are respectively inputto the image processing sections 301, 303, 309, and 311. Input 2K1Kpartial image data DIn1 to DIn4 are respectively segmented into fourpieces of image data. As shown in FIG. 10B, three pieces of image datafrom among segmented partial images are distributed to the respectiveimage processing sections so as to be adjacent to each other whenarranged as display image data DIn0 (hereinafter, also referred to asdistribution processing).

Specific description will be provided as to DIn1 input to the firstimage processing section 301 as an example. If DIn1 is input to thefirst image processing section 301, DIn1 is temporarily stored in theframe memory through the segmented video input section (see FIG. 5) inthe first image processing section 301. Thereafter, DIn1 is segmentedinto four pieces of partial image data by the data distribution controlsection. Thereafter, four pieces of data are also referred to assegmented partial image data DIn11, DIn12, DIn13, and DIn14 (see FIG.10B). The four pieces of segmented partial image data are obtained bybisecting (in total, quartering) partial image data DIn1 having theresolution of 2K1K in the vertical and horizontal directions. Thus, theresolution of each piece of segmented partial image data is “1K0.5K”.Then, in correspondence with the arrangement relationship of segmentedpartial image data in display image data DIn0, DIn11 is reserved in thefirst image processing section 301, and DIn12, DIn13, and DIn14 arerespectively distributed to the second image processing section 302, thethird image processing section 303, and the fourth image processingsection 304. The distribution is done such that the data distributioncontrol section transmits segmented partial image data to the imageprocessing sections 302, 305, and 306 through the bus 45 (see FIG. 3)which connects the image processing sections 301 to 316 to each other.

The distribution processing which has been described as to DIn1 input tothe first image processing section 301 as an example is similarlyperformed for partial image data DIn2 input to the third imageprocessing section 303, partial image data DIn3 input to the ninth imageprocessing section 309, and partial image data DIn4 input to the 11thimage processing section 311. That is, partial image data DIn1 to DIn4are further segmented into four pieces of partial image data, and fourpieces of partial image data are distributed to other image processingsections. Thus, even when the resolution of input display image dataDIn0 is low, image processing is performed using all the imageprocessing sections. In performing the image processing, the computersPC1 to PC4 are respectively connected to the image processing sections301, 303, 309, and 311. For this reason, in Step S230 of FIG. 9described above, the input mode control section 65 turns on the LEDs inthe connection terminals (in this example, the connection terminalscorresponding to the video input sections 201, 203, 209, and 211) whichconnect the projector PJ and the computers to each other.

Next, the image processing in each image processing section includingthe above-described distribution processing will be described withreference to a flowchart. The processing may be divided into processingby the image processing sections 301, 303, 309, and 311 (hereinafter,also referred to as main image processing sections) which directlyreceive partial image data DIn1 to DIn4 from the computers PC1 to PC4and processing by the image processing sections (hereinafter, alsoreferred to as sub image processing sections), which receive segmentedpartial image data from the main image processing sections, other thanthe image processing sections 301, 303, 309, and 311.

FIG. 11 is a flowchart illustrating the flow of image processing by themain image processing sections and the flow of image processing by thesub image processing sections. (A) in FIG. 11 shows the flow of imageprocessing by the main image processing sections, and (B) in FIG. 11shows the flow of image processing by the sub image processing sections.The image processing by the main image processing sections starts whenpartial image data DIn1 to DIn4 are respectively input from thecomputers PC1 to PC4. If partial image data is input to each main imageprocessing section (Step S310), the CPU of the main image processingsection stores partial image data in the frame memory through thesegmented video input section (see FIG. 5) and segments partial imagedata into four pieces of segmented partial image data by the datadistribution control section (Step S320). Thereafter, the CPUdistributes three pieces of segmented partial image data from amongsegmented partial image data to the sub image processing sections (StepS330). After the distribution processing, the CPU of the main imageprocessing section performs ambient pixel data exchange processingbetween segmented partial image data reserved in the frame memory andsegmented partial image data reserved or distributed in other imageprocessing section (Step S340). After the exchange of ambient pixeldata, filtering of segmented partial image data is performed using pixeldata (Step S350). After the filtering, in order to adjust 4K2K imagedata to the size of the display image in the 8K4K mode, resolutionconversion processing is performed for expanding the resolution twotimes (Step S360). As the resolution conversion processing, pixelinterpolation processing which has been generally used is performed.Specifically, bi-cubic convolution, bilinear interpolation, or the likeis used. After the resolution conversion processing, image data isoutput to the video composition section 40 (Step S370). In this way, theimage processing by the main image processing sections ends.

Meanwhile, in the sub image processing sections ((B) in FIG. 11), theimage processing starts simultaneously when the main image processingsections receive partial image data and is in the standby state untilthe segmented partial image data is received from the main imageprocessing sections (Step S430). If segmented partial image data isreceived from the main image processing sections (Step S430: YES),segmented partial image data is temporarily stored in the frame memoriesthrough the data distribution control section (Step S435). Aftersegmented partial image data is stored in the frame memories, ambientpixel data exchange processing is performed between segmented partialimage data and segmented partial image data reserved or distributed inother image processing sections (Step S440). After the exchange ofambient pixel data, filtering of segmented partial image data isperformed using pixel data (Step S450). After the filtering, theabove-described resolution conversion is performed (Step S460).Thereafter, segmented partial image data subjected to the resolutionconversion is output to the video composition section 40 (Step S410). Inthis way, the image processing by the sub image processing section ends.

The video composition section 40 receives segmented partial image datatransmitted from the main image processing sections ((A) in FIG. 11:Step S370) and segmented partial image data transmitted from the subimage processing sections ( )B) in FIG. 11: Step S470), sorts thearrangement of segmented partial image data, and performs imagecomposition processing including arrangement decision processing forrearranging image data such that segmented partial image data isdisplayed as display image data DIn0 when synchronously displayed (seeFIG. 6: Step S160). The subsequent processing by the video processingapparatus 100 is the same as the processing after Step S170 of FIG. 6,thus description thereof will be omitted. In this way, when the inputmode is changed from the 8K4K mode to the 4K2K mode, the respectiveimage processing sections change the data input method and processingmethod.

Next, description will be provided as to the input method and processingmethod of image data when the input mode is the 2K1K mode, that is, whenthe resolution of image data of display image data DIn0 is 2K1K. FIG. 12is an explanatory view illustrating the input method and processingmethod of partial image data which are changed with the change to the2K1K mode. When the resolution of display image data DIn0 is 2K1K, 2K1Kpartial image data DIn1 is input from the computer PC1 to the firstimage processing section 301. That is, with the change of the input modeby the user using the manipulation section Ctr, in the above-describedinput mode change processing of the projector PJ (see FIG. 9), the LEDof the connection terminal corresponding to the first image processingsection 301 from among the LEDs of the connection terminals is turned on(see FIG. 9: Step S230). The user confirms that the LED is turned on andconnects the computer PC1 and the video input section 201 (see FIG. 3)of the projector PJ. Thereafter, the user issues an instruction to startdisplay through the master computer PCM. The subsequent processing isthe same as in the above-described 4K2K mode, except for thedistribution method of partial image data DIn1, thus description will beprovided as to the distribution method of partial image data DIn1 in the2K1K mode.

As shown in FIG. 12, partial image data DIn1 having the resolution of2K1K is input from the computer PC1 to the first image processingsection 301. The CPU of the first image processing section 301temporarily stores input partial image data DIn1 in the frame memory,and the data distribution control section of the first image processingsection 301 segments partial image data DIn1 into four pieces ofsegmented partial image data. Then, three pieces of segmented partialimage data from among the four pieces of segmented partial image dataare respectively distributed to the image processing sections 303, 309,and 311. That is, in this processing step, each of the image processingsections 301, 303, 309, and 311 respectively store one piece ofsegmented partial image data in the corresponding frame memory (see FIG.5). Thereafter, the data distribution control section of each of thefour image processing sections further segments segmented partial imagedata stored in the corresponding frame memory into four pieces of imagedata. Thereafter, as shown in FIG. 12, image data (hereinafter, alsoreferred to as two-step segmented image data) obtained by furtherquartering segmented partial image data is distributed to the respectiveimage processing sections. That is, in the 2K1K mode, image data isdistributed in two steps. The subsequent image processing is the same asin the 4K2K mode, thus description thereof will be omitted. In this way,in the 2K1K mode, the image processing sections execute the input andprocessing of image data.

As described above, in the projector PJ of this example, regardless ofwhether the resolution of display image data DIn0 is high or low, imagedata is distributed to all the image processing sections, and the imageprocessing is performed in all the image processing sections. Whendisplay image data DIn0 has low resolution (in this example, 4K2K and2K1K), the processing in the image processing sections includes theresolution conversion processing (see (A) in FIG. 11: Step S360 and StepS460). The resolution conversion processing which is pixel interpolationprocessing, such as bi-cubic convolution or bilinear interpolation,imposes a heavy load on the image processing sections. In this example,even when display image data DIn0 having low resolution is input, imagedata is distributed to and processed in the image processing sections,such that the processing which imposes a heavy load is distributed inthe image processing sections and performed in parallel. Thus, in thisexample, the projector PJ can reduce the specification and performanceof hardware resources necessary for the image processing sections andcan consequently achieve reduction in cost compared to a projector inwhich the number of image processing sections for use in processing ischanged depending on the resolution of input display image data DIn0,for example, a projector in which, in the 8K4K mode, processing isperformed by 16 image processing sections and, in the 4K2K mode,processing is performed by four image processing sections depending onthe ratio of lowering of resolution. Even when the input mode is changedand the resolution of display image data DIn0 input to the projector PJis changed, output is made to the video composition section 40 after theresolution conversion in the image processing sections. Thus, the videocomposition section 40 can perform the same processing even when theinput mode is changed, and it is not necessary that the videocomposition section 40 includes a processing unit for each input mode.

As the correspondence relationship with the appended claims, the videoprocessing apparatus 100 corresponds to an image processing apparatusdescribed in the appended claims, the resolution 8K4K corresponds to areference resolution P described in the appended claims, and theresolution 4K2K and 2K1K correspond to a low resolution Q described inthe appended claims. In the 4K2K mode, the main image processing sectionand the sub image processing section correspond to a first imageprocessing section and a second image processing section described inthe appended claims.

In the 2K1K mode, the image processing section which inputs partialimage data DIn1 corresponds to a first image processing sectiondescribed in the appended claims, and the image processing section whichreceives segmented partial image data from other image processingsections correspond to a second image processing section described inthe appended claims, and the image processing section which receivestwo-step segmented image data from other image processing sectionscorresponds to a third image processing section described in theappended claims. In each input mode, the video input section connectedto the computer PC corresponds to an input section described in theappended claims.

B Modification

The invention is not limited to the above-described examples orembodiment, and may be carried out in various aspects without departingfrom the scope of the invention. For example, the followingmodifications may be made.

B1 Modification 1

In the above-described example, for example, the projector PJ inputs2K1K partial images as 16 pieces of partial image data DIn1 to DIn16 inthe 8K4K mode and inputs 2K1K partial image data as four pieces ofpartial image data DIn1 to DIn4 in the 4K2K mode. That is, although acase has been described where the resolution of one piece of partialimage data DIn is 2K1K, the invention is not limited thereto. Forexample, the resolution of one piece of partial image data DIn may bearbitrarily used in the segmentable range of display image data DIn0such that the resolution of one piece of partial image data DIn is“1K1K”, “1K0.5K”, or the like.

B2 Modification 2

Although in the above-described example, the projector PJ performs thedistribution processing of input partial image data DIn withoutdepending on the resolution of display image data DIn0, and the imageprocessing is performed in all the image processing sections, theinvention is not limited thereto. For example, in the 4K2K mode, inputpartial image data DIn may be processed in 12, 8, or 6 image processingsections in a distributed manner. That is, instead of decreasing thenumber of image processing sections to be used depending on the ratio oflowering of resolution of display image data DIn0 to a predeterminedresolution (in this example, 8K4K) as a reference, display image dataDIn0 is processed by using a larger number of image processing sectionsthan “the number of image processing sections according to the ratio oflowering of resolution”. Therefore, it is possible to reduce thespecification and performance of hardware resources necessary for therespective image processing sections compared to a projector in whichthe number of image processing sections to be used in processingdecreases depending on the ratio of lowering of the resolution of inputdisplay image data DIn0.

B3 Modification 3

Although in the above-described example, for example, in the 4K2K mode,2K1K partial image data DIn is input to the four main image processingsections, two-step segmented image data is distributed from the mainimage processing sections to the sub image processing sections, andafter ambient pixel data exchange processing between the imageprocessing sections, filtering and resolution conversion are performed,in Modification 3, the processing sequence may be changed. For example,in the 4K2K mode, after partial image data DIn1 to 4 are input to themain image processing sections (image processing sections 301, 303, 309,and 311), ambient pixel data exchange processing may be performedbetween the main image processing sections, filtering may be thenperformed, and subsequently partial image data may be segmented intotwo-step segmented image data and distributed to the sub imageprocessing sections. Thereafter, two-step segmented image data issubjected to resolution conversion in the image processing sections andoutput to the video composition section 40. If this processing sequenceis used, the image processing sections which perform the ambient pixeldata exchange processing may be limited to the main image processingsections. Even through this processing, the same effects as in theabove-described example can be obtained. In the 4K2K mode and the 2K1Kmode, before input partial image data DIn is segmented, input partialimage data may be segmented in the form including ambient pixelsnecessary for processing segmented partial image data and two-stepsegmented image data. This example is shown in FIG. 13. As shown in FIG.13, for example, in the 4K2K mode, when partial image data DIn1 input tothe image processing section 301 is quartered, instead of simplysegmenting partial image data into four pieces of image data, partialimage data is segmented so as to include ambient pixels. If partialimage data is segmented in this manner, it is possible to reduce ambientpixel data exchange processing. Even through this processing, the sameeffects as in the above-described example can be obtained.

B4 Modification 4

Although in the above-described example, this image processing method isused in the projector PJ, the invention is not limited thereto. Thisimage processing method can be applied to an image display apparatuswhich displays image data as images, such as a liquid crystaltelevision, a plasma television, an organic EL display, or an electronicpaper. The video processing apparatus 100 may be separated from theprojector or other video display apparatuses, or may be incorporatedinto the video display system. The video processing apparatus may beseparated from the video display apparatus, video data output from thevideo processing apparatus may be output to a plurality of video displayapparatus in a state of being segmented into multiple pieces of data,and display images displayed by the video display apparatuses may bedisplayed on a display surface in a unified manner. For example, displayimage data DIn0 subjected to video processing from the video processingapparatus is segmented into four pieces of image data, and respectivepieces of segmented image data are input to four projectors. Theprojectors project and display the segmented images on the single screenSC so as to be displayed as a single display image on the screen SC in aunified manner.

1. An image processing apparatus which processes image datacorresponding to a display image, the image processing apparatuscomprising: an input section which is configured to input at least imagedata corresponding to an image having a reference resolution P which isa resolution as a reference, the resolution representing the number ofconstituent pixels; M image processing sections which have a function ofprocessing image data of M (where M is an integer equal to or greaterthan 2) images segmented from the image having the reference resolutionP; an image processing control section which, when an imagecorresponding to input image data is the image having the referenceresolution P, causes the M image processing sections to process imagedata corresponding to the M partial images segmented from image dataand, when an image corresponding to input image data is an image whichhas a resolution Q lower than the reference resolution P and is of sizeto be processable by N (where N is an integer equal to or smaller thanM−1) image processing sections, causes L image processing sections fromamong the M image processing sections to process image datacorresponding to L (where L is an integer equal to or greater than N+1and equal to or smaller than M) partial images segmented from imagedata; and an image composition section which reconstructs the displayimage on the basis of image data corresponding to the partial imagessubjected to image processing by the M or L image processing sections.2. The image processing apparatus according to claim 1, wherein aplurality of input sections are provided to input image datacorresponding to the display image as multiple pieces of segmented imagedata.
 3. The image processing apparatus according to claim 1, whereinthe L image processing sections include a first image processing sectionwhich, before the image processing, receives image data input by theinput section, segments received image data into a predetermined numberof pieces of image data, reserves at least one piece of segmented imagedata for use in processing, and distributes unreserved segmented imagedata to other image processing section, and a second image processingsection which receives image data distributed by the first imageprocessing section.
 4. The image processing apparatus according to claim3, wherein the second image processing section segments image datareceived from the first processing section into a predetermined numberof pieces of image data, reserves at least one piece of segmented imagedata for use in processing, and distributes unreserved segmented imagedata to image processing sections other than the first and second imageprocessing sections, and the L image processing sections include a thirdimage processing section which receives image data distributed by thesecond image processing section.
 5. The image processing apparatusaccording to claim 1, wherein M and L are the same number.
 6. The imageprocessing apparatus according to claim 1, wherein the referenceresolution P is “8K4K”.
 7. A projection display apparatus which projectsa projected image on a projection surface on the basis of image datacorresponding to a display image, the projection display apparatuscomprising: the image processing apparatus according to claim
 1. 8. Aprojection display apparatus which projects a projected image on aprojection surface on the basis of image data corresponding to a displayimage, the projection display apparatus comprising: the image processingapparatus according to claim
 2. 9. A projection display apparatus whichprojects a projected image on a projection surface on the basis of imagedata corresponding to a display image, the projection display apparatuscomprising: the image processing apparatus according to claim
 3. 10. Aprojection display apparatus which projects a projected image on aprojection surface on the basis of image data corresponding to a displayimage, the projection display apparatus comprising: the image processingapparatus according to claim
 4. 11. A projection display apparatus whichprojects a projected image on a projection surface on the basis of imagedata corresponding to a display image, the projection display apparatuscomprising: the image processing apparatus according to claim
 5. 12. Aprojection display apparatus which projects a projected image on aprojection surface on the basis of image data corresponding to a displayimage, the projection display apparatus comprising: the image processingapparatus according to claim
 6. 13. A video display system comprising: avideo display apparatus which includes the image processing apparatusaccording to claim 1 and displays an image on the basis of image datacorresponding to a display image; and a storage which stores image datacorresponding to the display image and inputs image data correspondingto the display image to the video display apparatus.
 14. A video displaysystem comprising: a video display apparatus which includes the imageprocessing apparatus according to claim 2 and displays an image on thebasis of image data corresponding to a display image; and a storagewhich stores image data corresponding to the display image and inputsimage data corresponding to the display image to the video displayapparatus.
 15. A video display system comprising: a video displayapparatus which includes the image processing apparatus according toclaim 3 and displays an image on the basis of image data correspondingto a display image; and a storage which stores image data correspondingto the display image and inputs image data corresponding to the displayimage to the video display apparatus.
 16. A video display systemcomprising: a video display apparatus which includes the imageprocessing apparatus according to claim 4 and displays an image on thebasis of image data corresponding to a display image; and a storagewhich stores image data corresponding to the display image and inputsimage data corresponding to the display image to the video displayapparatus.
 17. A video display system comprising: a video displayapparatus which includes the image processing apparatus according toclaim 5 and displays an image on the basis of image data correspondingto a display image; and a storage which stores image data correspondingto the display image and inputs image data corresponding to the displayimage to the video display apparatus.
 18. A video display systemcomprising: a video display apparatus which includes the imageprocessing apparatus according to claim 6 and displays an image on thebasis of image data corresponding to a display image; and a storagewhich stores image data corresponding to the display image and inputsimage data corresponding to the display image to the video displayapparatus.
 19. An image processing method which processes image datacorresponding to a display image and processes at least image datacorresponding to an image having a reference resolution P which is aresolution as a reference, the resolution representing the number ofconstituent pixels, the image processing method comprising: when imagedata corresponding to the input display image is the image having thereference resolution P, segmenting image data into image datacorresponding to M (where M is an integer equal to or greater than 2)partial images having a resolution R and processing segmented image datain parallel; when image data corresponding to the input display imagehas a resolution Q lower than the reference resolution P, and one of N(where N is an integer equal to or smaller than M−1) segmented partialimages has a resolution equal to or lower than a resolution R,segmenting image data into L (where L is an integer equal to or greaterthan N+1 and equal to or smaller than M) pieces of image data andprocessing image data corresponding to the partial images in parallel;and reconstructing the display image on the basis of processed imagedata corresponding to the M or L partial images.
 20. A computer readablestorage medium storing a computer program which causes a computer torealize an image processing function of processing image datacorresponding to a display image and processing at least image datacorresponding to an image having a reference resolution P which is aresolution as a reference, the resolution representing the number ofconstituent pixels, the computer program causing the computer toexecute: when image data corresponding to the input display image is theimage having the reference resolution P, segmenting image data intoimage data corresponding to M (where M is an integer equal to or greaterthan 2) partial images having a resolution R and processing segmentedimage data in parallel; when image data corresponding to the inputdisplay image has a resolution Q lower than the reference resolution P,and one of N (where N is an integer equal to or smaller than M−1)segmented partial images has a resolution equal to or lower than aresolution R, segmenting image data into L (where L is an integer equalto or greater than N+1 and equal to or smaller than M) pieces of imagedata and processing image data corresponding to the partial images inparallel; and reconstructing the display image on the basis of processedimage data corresponding to the M or L partial images.